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Abstract 

This article presents an innovative framework that provides a means to understand and re¬ 
evaluate student course evaluation systems. We present three major concepts vital to course 
evaluation systems and explain how they inform five evolutionary stages. Additionally, we show 
how the major stakeholders - students, faculty and administrators - are impacted in each 
evolutionary stage. This framework allows for development of student course evaluation 
systems from one where procedures are simply followed, to one where feedback is available and 
used in real time. The framework also provides a means to critically evaluate and re-evaluate 
current and future course evaluation systems. 

Introduction 

Despite the fact that students evaluate courses on a regular basis, as frequently as each time a 
class is taught, the present process is cumbersome as an actual tool for improving instruction. 
Students view the evaluation process as one that yields small returns for the effort spent 
(Edstrom, 2008); faculty question the reliability and validity of results (Johnson et al., 2013); 
rates of returns are inconsistent and student comments may be haphazard (Anderson, 2008; 
Donmeyer & Chapman, 2004; Ory, 2001). Yet, administrators dutifully collect and use student 
course evaluations as a measure of quality for program accreditation and faculty retention, 
promotion, and tenure (RPT) reviews (Edstrom, 2008; Beran & Violato, 2005; Johnson et al., 
2013). The prevailing process of course evaluation is post hoc (Balam & Shannon, 2010) - that 
is, results are applied to courses and curricula after the students have moved on to other courses; 
students may use evaluations to retaliate and less often to review; and for faculty the 
consequences of a poor evaluation may be costly, as they provide little guidance for improving 
teaching and less recourse to counteract a negative review. 

While each of the primary stakeholders - students, faculty, and administrators - can appreciate 
the difficulty in developing effective evaluation systems, each also sees the potential value that 
may emerge from effective course feedback, (Gravestock & Gregor-Greenleaf, 2008, Reed et al., 
2012). The purpose of this article is to present an innovative framework to revitalize processes 
and procedures for evaluating classes. This system will move student feedback from one in 
which procedures are simply followed, to one where feedback is available and used in real time, 
using a purposive approach. It is organized around a framework that offers guidance and 
instructions for directing the actions of all involved. 

Background 

The framework has been tested over a period of eleven years, during which the feedback system 
was shepherded from paper to electronic formats, results were sequestered in departments to 
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web-based data displays, and through several boisterous policy and procedure revisions. As a 
result, in this article, the framework that was conceptualized is described through its evolution. 
We also describe how the stakeholders in student feedback systems created the primary concepts 
involved and how the innovative feedback system developed. 

A group of faculty, students, and administrators was committed to move a course feedback 
system from one that was characterized by cloistered results used for disparate purposes by 
groups who did not communicate to a system characterized by streamlined processes, and 
common-sense conversations about results held in open forums (cf. Reed et al., 2012). This 
framework enables exploring relationships between the perspectives and goals of the three major 
constituents in student course feedback systems in higher education, using three organizing 
concepts, clarity, visibility, and functionality. However, before moving to the framework, the 
explicit rights and responsibilities of the primary constituents regarding student feedback must be 
considered. 

Student rights and responsibilities in student course feedback programs 

From the student’s perspective, effective student course feedback provides a means to 
communicate anonymously to faculty, for the purposes of instructional improvement as well as 
simply voicing discontent. Many students take their evaluative role seriously, believing they are 
fair and that they can identify effective from ineffective teaching (Beran & Violato, 2005; 
Campbell & Bozeman, 2008; Harris & Twiname, 2012). However, students seek reassurance that 
administrators receive student ratings, since many students believe their evaluations are 
inconsequential and ignored when it comes to faculty promotions and raises, and in the case of 
poor teaching, censure and dismissal (Campbell & Bozeman, 2008; Harris & Twiname, 2012). 

Though the majority of enrolled students do not access results of evaluations, students express 
the desire to use results of previous students’ evaluations to select the best courses and 
instructors for their own plan of study (Gravestock & Gregor-Greenleaf, 2008). Commercial 
services such as Professor Recommendations (https://www.myedu.com/professor- 
recommendations/) and Rate My Professors (http://www.ratemyprofessors.com) are increasingly 
used (Zaragosa, 2007). These commercial services allow an alternative for student evaluations, 
yet also require additional evaluative effort by students, and may be less trustworthy than 
university-run processes, as anyone may rate a course without actually providing proof that they 
enrolled in, participated in, or completed a course. Students are interested in effective 
institutional feedback systems that offer ease of use and greater confidence in results. 

Faculty viewpoint and responsibilities in student course feedback programs 

From the faculty’s view, effective student feedback provides a means of improving teaching 
while building evidence for retention, promotion and tenure. If they gain new insight from 
course evaluations, faculty will modify instruction positively if they have already mastered 
content and have learned several different teaching strategies (Balam & Shannon, 2010; Benton, 
2011; Campbell & Bozeman, 2008). In institutions where student feedback is coupled with 
teacher development, improvement in student ratings may be seen over time (Barrie, Ginns, and 
Prosser, 2005; Murray, 1997; Reed et al., 2012). Unfortunately, in early career stages (e.g., pre¬ 
tenure probationary period), it is common for course feedback to be used to revise teaching 
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towards less desirable pedagogies in order to avoid low ratings and negative comments 
(Edstrom, 2008; Zhao & Gallant, 2012). Faculty also defend themselves against poor ratings and 
negative comments, to avoid difficulty during RPT reviews. Because of the energy expended on 
responding to poor feedback and the potential impact of ratings on teaching competencies and 
careers, faculty are interested in effective and fair institutional feedback systems. 

Administrators ’ rights and responsibilities in student course feedback programs 

Administrators consider effective student course feedback to be vital to institutional integrity. 
Because assessment of courses and instructors is required for accreditation, student feedback 
serves a critical audit function (Johnson et al., 2013; Zhao & Gallant, 2012). In many 
institutions, student course evaluations are mandated to occur at regular intervals, and the 
outcome of evaluations must be made available to students and be included in faculty files. 
Student course assessments are viewed as ratings of teaching abilities (Benton, 2011), though 
only extremely low ratings will trigger a formal investigation of teaching competence (Edstrom, 
2008). Primarily due to the audit function that feedback systems serve, yet also to maintain the 
school’s reputation, administrators at institutions of higher education are invested in maintaining 
effective and efficient student course feedback systems. 

In summary, the present system is one of competing agendas, distrust, and miscommunication. 
The three stakeholders in student course assessment regard student feedback as normative for 
different and disconnected reasons (Benton, 2011). While individually each stakeholder may 
work to use feedback for their own purposes, together they shy away from a shared responsibility 
for using course and instructor feedback to improve teaching and learning and resistance to 
changing feedback practices can be anticipated (Edstrom, 2008). Application of the conceptual 
framework described in this article offers an opportunity to make shared decisions about course 
feedback policy and procedures, ultimately aligning the goals and protecting the rights and 
responsibilities of all interested parties. 

Shared concepts in student feedback systems 

Three key concepts - clarity, visibility, and functionality - form the basis of student course 
feedback programs in higher education. These concepts remove much of the competing agendas, 
distrust, and miscommunication within systems in use while at the same time, push the 
evaluative system towards providing meaningful data that will encourage teaching excellence. 

Clarity. Clarity addresses reliability and validity of the data set, and of measurement procedures, 
so that results are believable and dependable. Formalization and publication of policy and 
procedures regarding administration of feedback systems is included in the concept of clarity 
(Benton & Cashin, 2014). Further, clarity includes resolving variation in interpretation and 
meaning of results for all stakeholders, as well as providing benchmarking, and ensuring the 
readability of results as they are presented to users (cf. Balam & Shannon, 2010; Benton & 
Cashin, 2014). For example, a consensus regarding the publication, interpretation, and use of 
results in courses with enrollments of less than 30 students can and should be achieved, at first 
for the institution as a whole and later for departments and programs. 
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Visibility. Visibility is providing results to the right people at the right time and in the best place; 
consequently, visibility means different things to students, who would like results “yesterday” 
(Balam & Shannon, 2010), faculty, who may seriously resist making comments public (Balam & 
Shannon, 2010), and administrators, who are cognizant of the costs involved in running robust 
systems (Benton, 2011). Formalization and publication of policy and procedure regarding 
display and use of feedback results are included in the concept of visibility. Visibility also 
includes publication and promulgation of the use of feedback by interested parties, as well as 
level and speed of access to results (Balam & Shannon, 2010). Access to results has been 
revolutionized with the move from paper-and-pencil systems (when results were often received 
after a delay of six months in large departments) to electronic, web-based systems where faculty 
can request and receive mid-course assessments in time to adjust the present course, to the 
delight of currently enrolled students. 

Functionality. All stakeholders have motives for feedback use; consequently, functionality 
becomes a third critical concept in student feedback programs. Functionality also includes 
regulations and requirements that direct processes around student course feedback data collection 
and display, and benchmarking. A critical piece of functionality is the development of policies 
and procedures that govern the use of results in RPT decisions (cf., Benton, 2011). A less 
inflammatory issue relative to functionality is the use of results in faculty development plans 
(Benton, 2011). Faculty and student governance structures are central to oversight of 
functionality when policy and procedures are written and implemented, while information 
technology support services are critical to the execution of online systems (Kinash et al., 2011). 

Implementing and revitalizing the course evaluation system 

Revitalizing the student course feedback system may occur slowly and in phases or stages as 
described here (Major & Major, 2011; Reed et al., 2012). As with all stage theory in complex 
systems, the level of revitalization reached in using these concepts is unpredictable and uneven 
and may, in fact, regress on occasion (Reed et al., 2012). For example, a course feedback 
system may be unusually sophisticated in its visual display of information, or visibility, and lag 
in the clarity of policies regarding the use of feedback, and in stakeholder understanding of the 
intended use of results, or functionality. Yet, these three concepts, clarity of processes, visibility 
of results, and functionality of feedback, present themselves along a continuum that can be 
roughly divided into five phases: from the initial, or Proto phase, through the Procedural, 
Motivational, and Relational phases, to culminate in the Social phase. 

The Proto Phase: Something Is In Place. The Proto phase is the least formed and least 
desirable of all the phases, but at least some form of course assessment is in place in most, but 
perhaps not all, departments. In the Proto phase, clarity within the system is characterized by 
decentralized interpretations of student course feedback results; for example, each department 
has its own rating scale so that Law professors may excel with a high of 7 while Science 
professors may only ever achieve a maximum score of 5. Quantitative and qualitative results are 
used for summative evaluative purposes, improving the course after the course is over - 
sometimes called the “autopsy” approach (Hmieleski & Champagne, 2000). Since procedures 
for data collection, interpretation, and use vary across instructional unit, multiple internal and 
external threats to reliability are present (e.g., Benton & Cashin, 2014). During the Proto phase, 
forms are paper-and-pencil and the person giving the surveys is occasionally the course 
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instructor. The instructor may feel free to alter responses, or simply not turn them in for 
processing. Visibility in the Proto phase is limited to instructors and administrators; long periods 
of delay occur while data are processed. Numerical results are available to a select group of 
students at the time of RPT reviews. Functionality in the Proto phase is almost completely 
disorganized and is focused on the use of feedback as a tool for rating instructor teaching 
effectiveness, as opposed the effectiveness of the course in meeting educational objectives. The 
use of feedback for RPT actions is not governed by university-level policy. 

Procedural Phase: Mechanical and Backstage. Matters will have improved in the Procedural 
phase. The Procedural phase of student course feedback programs addresses standardization of 
processes involved in administration of data collection forms, questions, and display of data (cf., 
Major & Major, 2011). Conversion to an online student feedback system solves the majority of 
data collection, standardization, and analysis problems, e.g., Law and Science instructor ranges 
and means are now available for comparison. Policies and procedures govern data access 
privileges and frequency of data collection, and dictate use by faculty and administrators. For 
many universities, the Procedural phase is the status quo for student feedback programs. 

Commonly, in the Procedural phase, policy and procedures remain static while survey questions 
are revised at a departmental or program level; university-level questions are seldom modified so 
that benchmarking functionality at a university level is preserved (e.g., when dispensing 
university-level teaching awards, it is helpful when comparing Law to Science that student 
feedback questions and metrics are similar across departments within the institution). Data may 
be mined for accreditation purposes, e.g., evidence that general education and specialty programs 
are meeting stated outcomes. 

Clarity in the Procedural phase is characterized by the presentation of results without 
interpretation. While feedback remains a summative (end-of-term and cross-sectional) 
operation, threats to reliability have been greatly decreased through online administration, 
common start and end dates for data collection, and common data reporting processes. Visibility 
in the Procedural phase becomes mandated while still limited, with results available after some 
processing delay. Not all stakeholders will receive the same report; some will receive 
frequencies and means and institution-wide metrics while others receive results as percentages of 
responses at the level of program only. Select students may receive comments at the time of 
instructor formal reviews. Functionality in the Procedural phase focuses on regulatory purposes, 
with time spent on formulating and revising institution-level policies and procedures, and the use 
of feedback in RPT actions. The intended use of results by all stakeholders is assumed and 
seldom examined. 

Motivational Phase: Galvanizing and Public. In the Motivation phase, student course feedback 
has gone public and early adopters of change processes have become galvanizing agents to 
improve the feedback system. Results may be displayed in more prominent and useful locations. 
Feedback displays are accompanied by information designed to improve meaning and 
interpretations. Policies and procedures are revised in open forums, including all stakeholders’ 
input. Expected developments in this Motivational phase include lively faculty discussions, and 
remarkable student interest in results, e.g., in our experience, moving a link to numerical results 
to the university class schedule resulted in a feedback page hit increase of 1000% in one term at 
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our institution. Administrators may use benchmarking data to set annual goals to meet or exceed 
department, college, or university averages. 

Clarity of feedback processes is characterized by the presentation of results with interpretations 
and benchmarks, though results remain largely used for summative purposes. Information about 
data meaning, readability, and interpretation is added to reports for students, instructors, and 
administrators. Students may appreciate a visual display of results to accompany numerical 
reports, as long as instructions for interpretation are included. For example, when assessing the 
efficacy of bar charts that accompanied numerical displays, we found that students in focus 
groups searched for courses with results displaying, “all tall bars.” Adding an explanation of 
how to read a bar chart solved the problem. 

Visibility in the Motivational phase is noticeably heightened, with policies and procedures 
written, rewritten, and publicly certified following vigorous - and sometimes tense - discussion 
in committees and plenary sessions. A communication plan regarding changes in the location of 
links to data and the data display will help, as long as it includes multiple outreach and vetting 
sessions to regulatory committees and governance bodies. 

Interestingly, functionality evolves from being regulatory, that is, regulating how feedback data 
are collected and analyzed, to being egoistic, with each stakeholder focusing on using feedback 
for their own unique purposes. Specifically, students value giving and receiving information, 
faculty value the benchmarking and development aspect of the data yet remain fearful of the 
weight of results in RPT decisions, and administrators value the developmental, benchmarking, 
and accreditation uses of feedback. The egoistic nature of functionality can be seen in the 
different reports that stakeholders may receive, and in the nature of the conversations that take 
place around intended use of results. 

Relational Phase: Connective and Reflective. The Relational phase is characterized by student 
feedback programs that are connective and reflective in processes and policies. Student feedback 
focuses on placing results in the contexts of time, instructor and student characteristics, 
disciplinary norms and expectations, and course requirements. Course and instructor results are 
linked and whenever possible, results are displayed and analyzed longitudinally. Feedback 
results are combined with other assessments to present a robust collection of information about 
teaching processes and student progression. 

In this phase, faculty, students, and administrators regularly converse about the uses and 
limitations of student feedback, both as an outcome measure for learning, and as an indicator of 
teaching effectiveness. Students access results to inform course choice as well as to begin 
learning course material. Faculty use results to make improvements in their instructional design 
and methods. Policy and procedures guide administrators’ use of feedback in personnel 
decisions. 

Clarity in the Relational phase is illustrated by processes that are reviewed annually for minor 
changes, resulting in institution-wide confidence in the stability and meaning of results. 

Feedback is solicited (and offered) at midterm as well as at end of term. Consequently, results 
are used for pseudo-formative purposes. The meaning of results is derived longitudinally and in 
contextual formats, i.e., tracking results over time becomes normative. Results are combined 
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with other indicators of teaching effectiveness, including student satisfaction, and learning 
engagement for a holistic assessment of courses, students, instructors, programs, etc. Visibility is 
encouraged in the Relational phase; results are linked between a course, students, and faculty. 

For the first time, student comments are made public in some form. The intended and actual use 
of results to inform course choice by students, instructional improvement by teachers, and 
decisions by administrators is openly exchanged. In the Relational phase, functionality is 
constructive. All stakeholders adhere to policy and procedures regarding the use of results. 
Stakeholders engage in regular, between-term conversations about the uses and limits of 
feedback as an outcome measure. For the first time, all stakeholders receive the same numerical 
report. 

Social Phase: Conversational and Interactive. The final phase is Social, characterized by 
conversations and interactions regarding how to work together to improve the educational 
experience, with course feedback as one tool. Feedback is solicited, offered and used to make 
improvements in real time, using an open and recursive process. The continual use of feedback 
to improve teaching becomes normative for the campus climate. Changes in instruction become 
fluid and collaborative, as students respond to changes and offer more feedback. When engaging 
in a range of interactive and collaborative forums about teaching, all stakeholders are present and 
results are included. 

Clarity in the Social phase is characterized by feedback processes that, similar to the Relational 
phase, are reviewed often enough so that institutional confidence in results is high and feedback 
is solicited continuously; consequently results are used for true formative assessment. The 
meaning of these data changes dynamically, as they are used in real time; changes in instruction 
become fluid and formative. Stakeholders routinely include results in a range of interactive and 
collaborative forums dedicated to teaching. Visibility in the Social phase becomes ubiquitous. 
Stakeholders routinely publish, access, and evaluate numerical and qualitative results as part of a 
culture of instructional assessment, planning and improvement. Functionality in the Social phase 
is developmental. Stakeholders engage in regular conversations during the academic year about 
the use of feedback in shaping higher education. 

Conclusions 

The characteristics of each of these five phases are summarized in Table 1. The three concepts, 
clarity, visibility, and functionality are shown, and how these may develop as institutions change 
their systems through procedural, motivational, and relational phases with the ultimate goal of 
moving the course evaluation system to the social phase (see Table 1). 

The advantages of moving at least into the Relational phase in student course feedback systems 
counteract all the difficulties enumerated in the article’s introduction. Students welcome the 
increased access to results that guide them in course choice, and the sense that their feedback is 
meaningfully used. Faculty employ results from current students to improve the current course, 
and use feedback as one of many means to build their teaching abilities, reducing student 
feedback as a source of fear during RPT reviews. Administrators can use feedback metrics to 
guide high-level decisions with confidence in results, focusing more on instructional programs 
and other university initiatives than on individual faculty. In short, attending clarity, visibility. 
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and functionality in the feedback system has the potential to revitalize the educational mission in 
significant ways for the three groups who care deeply about teaching and learning. 
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Table 1 

Relationship between concept and phase in student course feedback system implementation 


Conceptual 

Component 

Program 

Component 

3 hases with identifying hallmarks 

Proto Procedural Motivational Relational Social 

Clarity 

Policy and 

Locally 

Commonly Commonly 

Contextually Dynamically 


procedures 

stated 

stated interpreted 

placed changing 


Assessment, 

administration 

Summative Summative Summative 

Pseudo- Formative 

formative 

Visibility 

Availability of 

results 

Limited 

Mandated Heightened 

Integrated Ubiquitous 


Distribution 

of results 

Delayed 

Delayed Immediate 

Immediate Real time 


Type of results 

available 

Numbers 

''lumbers lumbers 

lumbers, lumbers and 

some comments 

comments 

Functionality Use of 

Paper and 

Dnline Dnline with 

Dnline with Dnline with 


technology 

pencil 

diagnostics 

diagnostics diagnostics 


Intended use 

of results 

Student¬ 

centric 

Regulatory Egoistic 

Constructive Developmental 
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